Training a CNN with pseudo ground truth for CT artifact reduction

ABSTRACT

Training a CNN with pseudo ground truth for CT artifact reduction is described. An estimated ground truth apparatus is configured to generate an estimated ground truth image based, at least in part, on an initial CT image that includes an artifact. Feature addition circuitry is configured to add a respective feature to each of a number, N, copies of the estimated ground truth image to create the number, N, initial training images. A computed tomography (CT) simulation circuitry is configured to generate a plurality of simulated training CT images based, at least in part, on at least some of the N initial training images. An artifact reduction circuitry is configured to generate a plurality of input training CT images based, at least in part, on the simulated training CT images. A CNN training circuitry is configured to train the CNN based, at least in part, on the input training CT images and based, at least in part, on the initial training images.

CROSS REFERENCE TO RELATED APPLICATION(S)

This application is a continuation of and claims the benefit of U.S.Nonprovisional patent application Ser. No. 16/201,186, filed Nov. 27,2018, that claims the benefit of U.S. Provisional Application No.62/590,966, filed Nov. 27, 2017, the entire disclosures of which arehereby incorporated by reference as if disclosed herein in theirentireties.

FIELD

The present disclosure relates to training a convolutional neuralnetwork (CNN), in particular to, training the CNN with a pseudo (i.e.,estimated) ground truth for CT (computed tomography) artifact reduction.

BACKGROUND

Artifacts resulting from features, e.g., metal objects, have been apersistent problem in CT (computed tomography) images over the last fourdecades. One approach to overcome their effects is to replace corruptprojection data with values synthesized from an interpolation scheme.Another approach includes reprojection of a prior image. Existingcorrection methods, including for example, an interpolation- andnormalization-based technique (“NMAR”), may not produce satisfactoryresults for some clinical applications. Residual image artifacts mayremain in challenging cases and, in some instances, new artifacts can beintroduced by the interpolation scheme itself. Thus, artifacts, e.g.,metal artifacts, continue to be a major impediment, particularly inradiation and proton therapy planning as well as in orthopedic imaging.

Currently, artifacts, e.g., metal artifacts, in CT images continue tohinder clinical diagnosis. Although a number of artifact reductiontechniques have been implemented over the past several years, challengesremain and sufficient image quality may not always be achieved. Forexample, radiation and proton therapy planning are particularlysensitive to errors in the CT images, since incorrect estimation of atreatment beam stopping power may result in under treatment and tumorrecurrence or unnecessary radiation to the surrounding healthy tissues.

SUMMARY

In some embodiments, a method for computed tomography (CT) artifactreduction is provided. The method includes generating, by an estimatedground truth apparatus, an estimated ground truth image based, at leastin part, on an initial CT image, the initial CT image including anartifact. The method further includes adding, by feature additioncircuitry, a respective feature to each of a number, N, copies of theestimated ground truth image to create the number, N, initial trainingimages. The method further includes generating, by a CT simulationcircuitry, a plurality of simulated training CT images based, at leastin part, on at least some of the N initial training images. Each of atleast some of the plurality of simulated training CT images contains atleast one respective simulated artifact. The method further includesgenerating, by artifact reduction circuitry, a plurality of inputtraining CT images based, at least in part, on the simulated training CTimages. The method further includes training, by a convolutional neuralnetwork (CNN)training circuitry, a CNN based, at least in part, on theinput training CT images and based, at least in part, on the initialtraining images.

In some embodiments of the method, generating the estimated ground truthimage includes generating, by the artifact reduction circuitry, anintermediate CT image based, at least in part, on the initial CT image,the intermediate CT image comprising a reduced artifact. The estimatedground truth image is generated based, at least in part, on theintermediate CT image.

In some embodiments, the method further includes extracting, by thefeature addition circuitry, a number, M, training patches from each ofat least some of the N initial training images. In these embodiments,the simulated training CT images are generated based, at least in part,on at least some of the M training patches.

In some embodiments, the method further includes validating, by the CNNtraining circuitry, the trained CNN based, at least in part, on theintermediate CT image. In some embodiments, the method may furtherinclude reducing, by the trained CNN, an actual artifact in an actual CTimage. In some embodiments of the method, the artifact reductioncircuitry corresponds to metal artifact reduction circuitry. In someembodiments of the method, the initial CT image is an output of a CTscanner configured to image an imaging object and the imaging objectcontains a metal implant.

In some embodiments, an apparatus for generating an estimated groundtruth image includes an artifact reduction circuitry configured toreceive an initial CT image and to generate an intermediate CT imagebased, at least in part, on the initial CT image. The initial CT imageincludes a major artifact. The intermediate CT image includes a reducedartifact. The estimated ground truth image is generated based, at leastin part, on the intermediate CT image.

In some embodiments, the apparatus may further includefiltering/denoising circuitry configured to at least one of filterand/or denoise the intermediate CT image. In some embodiments of theapparatus, the artifact reduction circuitry is metal artifact reductioncircuitry configured to perform metal artifact reduction. In someembodiments of the apparatus, the filtering/denoising circuitry isconfigured to perform one or more of low-pass filtering, segmentationand/or regional averaging of the intermediate CT image to generate theestimated ground truth image. In some embodiments of the apparatus, theinitial CT image is an actual CT image of an imaging object thatcontains a metal implant.

In some embodiments, a convolutional neural network (CNN) systemincludes an estimated ground truth apparatus, feature additioncircuitry, a first computed tomography (CT) simulation circuitry and aCNN training circuitry. The estimated ground truth apparatus isconfigured to generate an estimated ground truth image based, at leastin part, on an initial CT image. The initial CT image includes anartifact. The feature addition circuitry is configured to add arespective feature to each of a number, N, copies of the estimatedground truth image to create the number, N, initial training images. Thefirst CT simulation circuitry is configured to generate a plurality ofsimulated training CT images based, at least in part, on at least someof the N initial training images. Each of at least some of the pluralityof simulated training CT images contains at least one respectivesimulated artifact. The first artifact reduction circuitry is configuredto generate a plurality of input training CT images based, at least inpart, on the simulated training CT images. The CNN training circuitry isconfigured to train the CNN based, at least in part, on the inputtraining CT images and based, at least in part, on the initial trainingimages.

In some embodiments of the system, the estimated ground truth apparatusincludes a second artifact reduction circuitry configured to generate anintermediate CT image based, at least in part, on the initial CT image.The intermediate CT image includes a reduced artifact. The estimatedground truth image is generated based, at least in part, on theintermediate CT image.

In some embodiments of the system, the feature addition circuitry isconfigured to extract a number, M, training patches from each of atleast some of the N initial training images. The simulated training CTimages are generated based, at least in part, on at least some of the Mtraining patches. In some embodiments of the system, the CNN trainingcircuitry is configured to validate the trained CNN based, at least inpart, on the intermediate CT image. In some embodiments of the system,at least one of the first and second artifact reduction circuitry ismetal artifact reduction circuitry configured to perform metal artifactreduction. In some embodiments of the system, the trained CNN isconfigured to reduce an actual artifact in an actual CT image. In someembodiments of the system, the initial CT image is an output of a CTscanner configured to image an imaging object, the imaging objectcontaining a metal implant. In some embodiments of the system, theinitial CT image is simulated based, at least in part, on a phantomimage.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings show embodiments of the disclosed subject matter for thepurpose of illustrating features and advantages of the disclosed subjectmatter. However, it should be understood that the present application isnot limited to the precise arrangements and instrumentalities shown inthe drawings, wherein:

FIG. 1 illustrates a functional block diagram of an estimated groundtruth apparatus consistent with several embodiments of the presentdisclosure;

FIG. 2A illustrates a functional block diagram of a convolutional neuralnetwork (CNN) training system consistent with several embodiments of thepresent disclosure;

FIG. 2B illustrates a functional block diagram of a trained CNN systemconsistent with several embodiments of the present disclosure;

FIG. 3 is a flowchart of example estimated ground truth image generationoperations consistent with several embodiments of the presentdisclosure;

FIG. 4 is an example flowchart of convolutional neural network (CNN)operations consistent with several embodiments of the presentdisclosure; and

FIGS. 5A through 5E illustrate a reference CT image, an uncorrected CTimage, a CNN input CT image and two CNN output images for one examplewith a phantom image input.

DETAILED DESCRIPTION

There are a number of classes of artifact reduction (e.g., metalartifact reduction (MAR)) techniques, with projection completion beingrelatively widely developed. Generally, these techniques are configuredto replace corrupt sinogram data in the artifact, e.g., metal, tracewith data synthesized by an interpolation technique, reprojection from aprior image or a combination of both that includes normalization. Oneexample technique is normalized metal artifact reduction (“NMAR”). Otherclasses of artifact reduction methods include scan acquisitionimprovement, physics-based preprocessing, iterative reconstruction andimage postprocessing. While image postprocessing algorithms have hadsome success, they are more useful when combined with sinogram domaincorrection. The current clinical techniques may fall short in providingrequisite image quality for the most demanding applications,particularly in radiation and proton therapy planning.

Deep learning may provide a solution to the long-standing artifactreduction problem. Deep learning, including for example convolutionalneural networks (CNNs), has been successfully applied to medical imageprocessing and analysis tasks. Generally, training a CNN includesproviding ground truth artifact-free images that may be used as networklabels (i.e., training images). Typically, a ground truth artifact-freeimage is configured to be identical to an input image except for anartifact present in the input image and that the CNN is to be trained toremove. Ground truth artifact-free images may not be available in someclinical cases. For example, it may not be possible to capture artifactfree images of a patient who has a metal implant. As used herein, metalimplants may include, but are not limited to, artificial joints, toothfillings, venous clips, etc.

Generally, the present disclosure relates to generating a pseudo, i.e.,estimated, ground truth image from a feature (i.e., artifact)-affectedimage. The generated estimated ground truth image may then be utilizedto perform supervised learning. The estimated ground truth image isgenerated, based at least in part, on an initial CT image that mayinclude an artifact (e.g., a metal artifact). In some situations, theartifact may be substantial, i.e., major. In some embodiments, anartifact reduction technique may be performed on the initial CT image toproduce an intermediate CT image that includes a reduced artifact priorto generating the estimated ground truth image. The estimated groundtruth image may then be utilized to train a CNN, for example, insituations where actual ground truth, artifact-free data is notavailable.

In an embodiment, the estimated ground truth image may be generatedbased, at least in part, on an initial CT image. The initial CT imagemay be output from a CT scanner or CT simulation circuitry. Theestimated ground truth may be generated based, at least in part, on afeature (e.g., metal)-affected image by filtering, segmentation andregion averaging. Features and/or nodules may then be randomly added tothe estimated ground truth image to create a set of training images, aswill be described in more detail below.

An artificial neural network (ANN) is a network of elements (e.g.,nodes) configured to receive input, change their internal state(activation) according to that input, and produce output depending onthe input and activation. The network is formed by connecting the outputof selected nodes to the input of other nodes to form a directed,weighted graph. The weights as well as the functions that compute theactivation can be modified by learning (e.g., training).

A deep neural network is an ANN that has a plurality of layers betweenthe input and output layers. A relationship between the input and theoutput may be linear or non-linear. A convolutional neural network (CNN)is a type of deep, feed-forward ANN, that includes one or moreconvolutional layers with fully connected layers on top. A multilayerperceptron (MLP) is a type of feed-forward ANN that includes at leastthree layers of nodes and each node, except for the input nodes, uses anonlinear activation function. An MLP may be trained using backpropagation, a supervised learning technique. The multiple layers andnon-linear activation of an MLP distinguish it from a linear perceptron.CNNs are a type of deep ANN that use a variation of multilayerperceptrons designed to use minimal preprocessing.

Deep learning is a type of machine learning technique that uses acascade of a plurality of layers of nonlinear processing units forfeature extraction and transformation. Each successive layer uses theoutput from the previous layer as input. Deep learning techniques learnin supervised (e.g., classification) and/or unsupervised (e.g., patternanalysis) manners. Deep learning algorithms learn multiple levels ofrepresentations that correspond to different levels of abstraction. Inother words, deep-learning methods are representation-learning methodswith multiple levels of representation, obtained by composing simple butnon-linear modules that each transform the representation at one levelinto a representation at a higher, slightly more abstract level. Withthe composition of enough such transformations, very complex functionscan be learned.

Generally, this disclosure relates to a training a CNN with an estimatedground truth image for CT artifact reduction. An apparatus, methodand/or system is configured to generate an estimated ground truth imagebased, at least in part, on an initial CT image that may contain anartifact. The apparatus, method and/or system is further configured totrain a CNN based, at least in part, on the estimated ground truthimage. For example, a plurality of features and/or nodules may be addedto the estimated ground truth image to generate a plurality of trainingimages that may then be used to train the CNN. The apparatus, methodand/or system may be further configured to perform artifact reduction onthe training images prior to provision to the CNN for training.Advantageously, the estimated ground truth image is configured tocorrespond to an actual imaging object, where a feature (e.g.,metal)-free variant is not available, e.g., a patient.

In an embodiment, a convolutional neural network (CNN) system includesan estimated ground truth apparatus, feature addition circuitry, a firstcomputed tomography (CT) simulation circuitry and a CNN trainingcircuitry. The estimated ground truth apparatus is configured togenerate an estimated ground truth image based, at least in part, on aninitial CT image. The initial CT image includes an artifact. The featureaddition circuitry is configured to add a respective feature to each ofa number, N, copies of the estimated ground truth image to create thenumber, N, initial training images. The first CT simulation circuitry isconfigured to generate a plurality of simulated training CT imagesbased, at least in part, on at least some of the N initial trainingimages. Each of at least some of the plurality of simulated training CTimages contains at least one respective simulated artifact. The firstartifact reduction circuitry is configured to generate a plurality ofinput training CT images based, at least in part, on the simulatedtraining CT images. The CNN training circuitry is configured to trainthe CNN based, at least in part, on the input training CT images andbased, at least in part, on the initial training images. The trained CNNmay then be used to reduce an actual artifact in an actual CT image.

FIG. 1 illustrates a functional block diagram 100 of an estimated groundtruth apparatus consistent with several embodiments of the presentdisclosure. The estimated ground truth apparatus 100 includes artifactreduction circuitry 106 and filter/denoising circuitry 108. Theestimated ground truth apparatus 100 is configured to receive an initialCT image 105. In one nonlimiting example, artifact reduction circuitry106 may correspond to metal artifact reduction (MAR) circuitry.

In an embodiment, the initial CT image 105 may correspond to an actualCT image 105A. For example, the actual CT image 105A may be provided byand/or received from CT scanner 114. CT scanner 114 may be configured tocapture projection data related to an imaging object 112 and to producea corresponding actual CT image 105A. If the imaging object 112 containsan artifact-producing feature, e.g., a metal implant, then the actual CTimage 105A may include an artifact, e.g., a metal artifact.

The actual CT image 105A may thus correspond to the imaging object 112(e.g., a patient). For example, the patient may have one or moreartifact-producing features, e.g., metal implants, as described herein.In this embodiment, estimated ground truth apparatus 100 is configuredto generate the estimated ground truth image that may then be utilizedfor training a CNN, as described herein.

In another embodiment, the initial CT image 105 may correspond to asimulated CT image 105B. The simulated CT image may be provided byand/or received from CT simulation circuitry 104. For example, thisembodiment may be configured to illustrate a proof of concept. Inanother example, this embodiment may support development of Artifactreduction circuitry 106 and/or filter/denoising circuitry 108. In thisembodiment, a phantom source circuitry 102 may be configured to generatea phantom image. The phantom image may contain at least one metalfeature corresponding to at least one metal implant. In one nonlimitingexample, phantom source circuitry 102 may be configured to generate aShepp-Logan type phantom. As is known, a Shepp-Logan phantom is astandard test image corresponding to a model of a human head that may beused in the development and testing of image reconstruction techniques.The Shepp-Logan phantom may incorporate radiation attenuation propertiesof a head and brain.

CT simulation circuitry 104 is configured to receive the phantom imagefrom phantom source circuitry 102. CT simulation circuitry 104 isconfigured to generate the simulated CT image 105B based, at least inpart, on the phantom image. The simulated CT image 105B is configured tocontain a major artifact. In one nonlimiting example, the major artifactmay correspond to a metal implant, as described herein. In onenonlimiting example, CT simulation circuitry 104 may correspond to“CatSim”, a computer assisted tomography simulation environment (GeneralElectric Global Research Center, Niskayuna, N.Y.). For example, CTsimulation circuitry 104 may be configured to implement filtered backpropagation (FBP) to generate the initial simulated CT image from theprovided phantom image.

In these embodiments, an actual ground truth image may not be available.The phantom image may be configured to contain a metal artifactcorresponding to a metal implant. The imaging object 112 may contain anartifact-producing feature, e.g., a metal implant. Thus, the initial CTimage 105 may contain an artifact. The artifact may be substantial,i.e., may be a “major” artifact.

Estimated ground truth apparatus 100 is configured to receive theinitial CT image 105. For example, Artifact reduction circuitry 106 maybe configured to receive the initial CT image 105 (simulated 105B oractual 105A) from CT simulation circuitry 104 or CT scanner 114.Artifact reduction circuitry 106 is configured to generate anintermediate CT image 107 based, at least in part, on the initial CTimage 105. The intermediate CT image 107 is configured to contain areduced artifact (i.e., “minor artifact”) compared to the major artifactcontained in the initial CT image 105. In one nonlimiting example,artifact reduction circuitry 106 may be configured to implement anormalized metal artifact reduction (NMAR) technique. The NMAR techniqueincludes segmenting metal artifacts in the image domain by thresholding.A three-dimensional forward projection may be configured to identify themetal trace in an original projection. Prior to interpolation, theprojections may be normalized based on a three-dimensional forwardprojection of a prior image. The prior image may be obtained, forexample, by a multi-threshold segmentation of the initial image. Theoriginal raw data are divided by the projection data of the prior imageand, after interpolation, denormalized again.

In other words, in the NMAR technique, artifact reduction circuitry 106may be configured to obtain a metal image and a prior image based, atleast in part, on the initial CT image by thresholding. Artifactreduction circuitry 106 may be configured to generate correspondingsinograms by forward projection. Artifact reduction circuitry 106 may befurther configured to generate an initial sinogram corresponding to theinitial CT image. The initial sinogram may then be normalized bydividing it by the corresponding prior sinogram. Artifact reductioncircuitry 106 may then be configured to utilize metal projections todetermine where data in the normalized sinogram are replaced byinterpolation. The interpolated and normalized sinogram may then bedenormalized by multiplying it with the sinogram of the prior imageagain. Artifact reduction circuitry 106 may then be configured toperform reconstruction on the denormalized sinogram to generate theintermediate CT image 107. Thus, Artifact reduction circuitry 106 may beconfigured to perform an initial correction via interpolation based, atleast in part, on the initial CT image 105 to yield the intermediatesimulated CT image 107. A corresponding intermediate CT image 107 isconfigured to have a reduced artifact compared to the initial CT image105.

Filter/denoising circuitry 108 is configured to receive the intermediateCT image 107 and to generate an estimated ground truth image 109.Filter/denoising circuitry 108 may be configured to perform one or moreof low-pass filtering, segmentation, regional averaging, etc., on theintermediate CT image 107. The estimated ground truth image 109 may thencontain relatively few artifacts compared to the intermediate CT image107 and to the initial CT image 105.

Thus, an estimated ground truth image may be generated by estimatedground truth apparatus 100 based, at least in part, on an initial CTimage that includes an artifact (e.g., a metal artifact) correspondingto a feature (e.g., a metal feature). The estimated ground truth imageis configured to contain relatively few and/or minor artifacts while theinitial CT image may contain a relatively major artifact. In otherwords, the phantom image and/or the imaging object may to correspond toa patient with a metal implant.

The estimated ground truth image may then be utilized to generate aplurality of training CT images. The training CT images may then beutilized to train a CNN, as will be described in more detail below.

FIG. 2A illustrates a functional block diagram 200 of a CNN trainingsystem consistent with several embodiments of the present disclosure.CNN training system 200 includes feature addition circuitry 202, CTsimulation circuitry 204, artifact reduction circuitry 206, CNN trainingcircuitry 208 and CNN 210. Feature addition circuitry 202 is configuredto receive an estimated ground truth image from, for example, estimatedground truth apparatus 100 of FIG. 1 . Feature addition circuitry 202 isconfigured to create a number, N, initial training images based, atleast in part, on the estimated ground truth image. For example, featureaddition circuitry 202 may be configured to generate N copies of theestimated ground truth image. Feature addition circuitry 202 may then beconfigured to add at least one respective feature to each copy of theestimated ground truth image. For example, a plurality of featuresand/or nodules may be randomly added to a plurality of copies of oneestimated ground truth image to generate a relatively large number ofinitial training images. In one nonlimiting example, the large numbermay be on the order of 10,000. In this manner, feature additioncircuitry 202 may be configured to generate N initial training images.In other words, based, at least in part, on one estimated ground truthimage, a plurality of initial training images may be generated. Theplurality of initial training images may be configured to correspond toimaging object 112 of FIG. 1 .

In some embodiments, feature addition circuitry 202 may be furtherconfigured to extract a number, M, training patches from each of atleast some initial training images. Each training patch corresponds to aportion of an initial training image. In one nonlimiting example, for aninitial training image of size 512×512 pixels, one or more image patchesof size 32×32 pixels may be extracted. Of course, training images ofother sizes and/or patches of other sizes are fully contemplated herein.The M training patches, N initial training images and/or a combinationthereof may then be utilized to train CNN 210, as described herein.

CT simulation circuitry 204 may be configured to receive N initialtraining images, M training patches for each of at least some of the Ninitial training images and/or a combination thereof. CT simulationcircuitry 204 may then be configured to generate a number, e.g., P,simulated training CT images based, at least in part, on at least someof the N initial training images and/or based, at least in part, on atleast some of the M training patches. The number P is less than or equalto N×M. In one nonlimiting example, P may be on the order of 10,000.Each of at least some of the simulated training CT images may contain atleast one respective simulated artifact.

Artifact reduction circuitry 206 may be configured to receive thenumber, P, simulated training CT images 205. Artifact reductioncircuitry 206 may then be configured to generate a plurality of inputtraining CT images (i.e., input training CT image data 207). Forexample, Artifact reduction circuitry 206 may be configured to performforward projection and/or reconstruction on the simulated training CTimages to generate the input training CT image data. The input trainingCT image data may then correspond to input data 207 to CNN 210. Thus,input data 207 to CNN 210 corresponds to the estimated ground truthimage with features and/or nodules added and after at least someartifact reduction operations. The input data 207 is configured toinclude a number, e.g., Q, of input training CT images generated based,at least in part, on one estimated ground truth image 109.

Thus, a plurality of initial training images and/or training patches maybe generated by feature addition circuitry 202. The plurality (e.g.,number P) of initial training images and/or training patches correspondto label data 203. A corresponding plurality of input training CT imagesand/or input training patches (i.e., input training CT image data) maybe generated by artifact reduction circuitry 206 based, at least inpart, on as few as one estimated ground truth image. The input trainingCT image data corresponds to input data 207. The label data 203 andinput data 207 may then be provided to CNN training circuitry 208.

CNN training circuitry 208 is configured to manage training operationsof CNN 210. Thus, CNN training circuitry 208 is configured to receivelabel data 203 and input data 207. Label data corresponds to trainingimages and/or training patches generated based, at least in part, on theestimated ground truth image and with added features and/or nodules. CNNtraining circuitry 208 may then be configured to provide input data 207to CNN 210 and to receive CNN output data 211 from CNN 210. CNN 210 isconfigured to generate CNN output data 211 based, at least in part, oninput data 207. CNN training circuitry 208 may then be configured toadjust parameters associated with CNN 210 to minimize a differencebetween CNN output data 211 and label data 203. Thus, CNN 210 may betrained based, at least in part, on initial training images and/ortraining patches and based, at least in part, on input training CT imagedata. The training is meant to configure CNN 210 to output a CNN outputCT image that corresponds to a CNN input CT image with artifacts, ifany, reduced, minimized or removed. In other words, during training,artifact reduction circuitry 206 is configured to reduce artifacts(e.g., metal artifacts) in simulated training CT images. During normaloperation, after training, artifact reduction circuitry may beconfigured to reduce artifacts in an actual initial CT image. An outputof the artifact reduction circuitry may then be provided to CNN 210 asinput data. CNN 210 may then be configured to further reduce artifactsand to provide CNN output data that corresponds to an actual CT image ofan imaging object with, for example, metal implants present but withassociated metal artifacts reduced.

In some embodiments, CNN training circuitry 208 may then be configuredto validate trained CNN 210 based, at least in part, on intermediate CTimages (i.e., output of artifact reduction circuitry 106) from estimatedground truth apparatus 100 of FIG. 1 . Thus, CNN training circuitry 208may be configured to provide input data to CNN 210, retrieve CNN outputdata and compare the CNN output data to the intermediate CT image data.

Thus, an estimated ground truth image may be generated based, at leastin part, on an initial CT image that may include artifacts, e.g., metalartifacts. The initial CT image may correspond to an imaging object,e.g., patient, that contains at least one artifact producing feature,e.g., a metal implant. The estimated ground truth image may thencorrespond to the imaging object.

The estimated ground truth image may then be utilized to generate aplurality of initial training images and/or training patches thatinclude respective features and/or nodules. The initial training imagesand/or training patches may then be used to train a CNN, e.g., CNN 210.The trained CNN may then be utilized to perform artifact reduction onactual CT image data. The actual CT image may be captured from theimaging object.

FIG. 2B illustrates a functional block diagram 220 of a trained CNNsystem consistent with several embodiments of the present disclosure.The trained CNN system 220 includes a trained CNN 210′, trained asdescribed herein. The trained CNN 210′ is configured to receive anactual CT image 207′. The actual CT image 207′ may be received, forexample, from CT scanner 114 of FIG. 1 , and may correspond to imagingobject 112. The imaging object 112 may include a feature that results inactual CT image 207′ containing an artifact. The trained CNN 201′ isconfigured to provide as output a reduced artifact image 211′ based, atleast in part, on the actual CT image 207′ input. Thus, the trained CNNmay be used to perform artifact reduction on actual CT image data.

Thus, a convolutional neural network (CNN) may be trained for artifactreduction of computed tomography (CT) images without a genuine groundtruth. In many clinical cases, CT scans of a patient without features,e.g., their metal implants, may not be available. For optimal CNNtraining for the purpose of artifact reduction, the label data should beidentical to the input image except for the artifacts that need to beremoved. An estimated ground truth may be generated from thefeature-affected image. In an embodiment, a feature-affected image maybe corrected by an interpolation technique (e.g., NMAR). The estimatedground truth is configured to contain relatively few artifacts and isconfigured to serve as the basis for the CNN labels. Nodules/featurescan be randomly added to this one estimated ground truth image togenerate many different samples for training. These images havefeatures, e.g., metal, added back in and are forward projected andreconstructed with NMAR to create the input for the CNN, which containartifacts. Patches can be extracted from full size 512×512 images toincrease the number of training samples. The input for the CNN mayinclude many simulated training CT images that each contain one or moreartifacts corresponding to the added features. The CNN may thus betrained on a relatively large number of training CT images generatedbased, at least in part, on one estimated ground truth image. Theestimated ground truth image may be specific to the imaging object,enhancing accuracy of the artifact reduction.

FIG. 3 is a flowchart 300 of example estimated ground truth generationoperations consistent with several embodiments of the presentdisclosure. In particular, flowchart 300 illustrates generating anestimated ground truth image based, at least in part, on an initial CTimage that may include a metal artifact. The operations of flowchart 300may be performed by, for example, estimated ground truth apparatus 100(e.g., artifact reduction circuitry 106 and filter/denoising circuitry108) of FIG. 1 .

In some embodiments, operations of flowchart 300 may begin withgenerating a phantom image including a metal feature at operation 302.For example, the phantom image may correspond to a Shepp-Logan typephantom (i.e., simulating a human head). In these embodiments, estimatedground truth generation apparatus may be utilized for developmentpurposes. Operation 304 may include generating an initial CT image thatmay include a major artifact. In an embodiment, the initial CT image maybe generated based, at least in part, on the phantom image that includesa feature. In another embodiment, the initial CT image may be generatedbased, at least in part, on an actual CT image of an imaging object thatincludes a feature, e.g., a metal implant.

Operation 306 may include generating an intermediate CT image with areduced artifact. For example, generating the intermediate CT image mayinclude performing an initial correction via, for example,interpolation. Operation 308 may include generating an estimated groundtruth image. For example, the estimated ground truth image may begenerated based, at least in part, on the intermediate CT image.Operation 308 may include, for example, low-pass filtering, segmentationand/or regional averaging, etc.

Thus, an estimated ground truth image may be generated based, at leastin part, on an initial CT image. The initial CT image may correspond toCT scanner output associated with an imaging object that may contain afeature, e.g., a metal implant, or a generated phantom image thatincludes a feature.

FIG. 4 is an example flowchart 400 of convolutional neural network (CNN)operations consistent with several embodiments of the presentdisclosure. In particular, the flowchart 400 illustrates utilizing anestimated ground truth image to train a CNN. Flowchart 400 may furtherillustrate using the trained CNN for artifact reduction. The operationsof flowchart 400 may be performed by, for example, CNN training system200 (e.g., feature addition circuitry 202, CT simulation circuitry 204,artifact reduction circuitry 206, CNN training circuitry 208, CNN 210and/or CNN 210′) of FIGS. 2A and/or 2B.

Operations of flowchart 400 may begin with providing and/or receiving anestimated ground truth image at operation 402. For example, theestimated ground truth image may be received from and/or provided by anestimated ground truth apparatus. Operation 404 may include adding oneor more features to create a number, N, initial training images. In someembodiments, operation 406 may include extracting a number, M, trainingpatches from one or more of the initial training images. A plurality ofsimulated training CT images may be generated at operation 408. At leastsome of the plurality of simulated training CT images may contain atleast one respective simulated artifact. A plurality of input trainingCT images may be generated at operation 410. The CNN may be trainedbased, at least in part, on initial training images and/or trainingpatches and based, at least in part, on input training CT images atoperation 412. In some embodiments, the trained CNN may be validatedbased, at least in part, on intermediate CT images with reducedartifacts at operation 414. In some embodiments, the trained CNN may beutilized to reduce an artifact in an actual CT image. For example, theactual CT image may contain an artifact associated with a featurecontained in an imaging object.

Thus, a CNN and may be trained based, at least in part, on an estimatedground truth image. Advantageously, the estimated ground truth image isconfigured to correspond to an actual imaging object, e.g., a patient,where a feature-free (e.g., metal-free) variant is not available.

EXAMPLE

FIGS. 5A through 5E illustrate a reference CT image 502, an uncorrectedCT image 504, a CNN input CT image 506 and two CNN output images 508,510 for one example with a phantom image input. Shepp-Logan typephantoms were used for this example. The phantoms were provided to CTsimulation circuitry, e.g., CatSim (CatSim, a computer-assistedtomography simulation environment, General Electric Global ResearchCenter, Niskayuna, N.Y.). Two CNNs were trained with more than 10,000patches extracted from the generated images to learn how to reduce metalartifacts. The networks were trained over 40 epochs with a learning ratestarting at 1e-3 and decreasing by a factor of the square root of theepoch. The loss function of the CNN-MSE (mean squared error) network isoptimized by the pixel-by-pixel mean squared error (MSE). The lossfunction of the CNN-WGAN (Wasserstein Generative AdversarialNetwork)-VGG (Visual Geometry Group) network is optimized in the WGANframework by the perceptual loss in the feature space from a pretrainedVGG network.

FIGS. 5A through 5E illustrate the results of the CNNs when the originalNMAR-corrected image is the input for validation. FIG. 5A is a referenceinput 502. FIG. 5B is an uncorrected initial CT image 504. FIG. 5C is anintermediate CT image 506 (i.e., output of artifact reduction circuitryconfigured to implement NMAR). FIG. 5D is a CNN output 508 for a CNNconfigured to optimize a loss function with mean squared error (MSD).FIG. 5E is a CNN output 510 for a CNN configured for WGAN with apre-trained VGG network.

As used in any embodiment herein, the term “logic” may refer to an app,software, firmware and/or circuitry configured to perform any of theaforementioned operations. Software may be embodied as a softwarepackage, code, instructions, instruction sets and/or data recorded onnon-transitory computer readable storage medium. Firmware may beembodied as code, instructions or instruction sets and/or data that arehard-coded (e.g., nonvolatile) in memory devices.

“Circuitry”, as used in any embodiment herein, may include, for example,singly or in any combination, hardwired circuitry, programmablecircuitry such as computer processors including one or more individualinstruction processing cores, state machine circuitry, and/or firmwarethat stores instructions executed by programmable circuitry. The logicmay, collectively or individually, be embodied as circuitry that formspart of a larger system, for example, an integrated circuit (IC), anapplication-specific integrated circuit (ASIC), a field-programmablegate array (FPGA), a programmable logic device (PLD), a complexprogrammable logic device (CPLD), a system on-chip (SoC), etc.

Embodiments of the operations described herein may be implemented in acomputer-readable storage device having stored thereon instructions thatwhen executed by one or more processors perform the methods. Theprocessor may include, for example, a processing unit and/orprogrammable circuitry. The storage device may include a machinereadable storage device including any type of tangible, non-transitorystorage device, for example, any type of disk including floppy disks,optical disks, compact disk read-only memories (CD-ROMs), compact diskrewritables (CD-RWs), and magneto-optical disks, semiconductor devicessuch as read-only memories (ROMs), random access memories (RAMs) such asdynamic and static RAMs, erasable programmable read-only memories(EPROMs), electrically erasable programmable read-only memories(EEPROMs), flash memories, magnetic or optical cards, or any type ofstorage devices suitable for storing electronic instructions.

What is claimed is:
 1. A method for computed tomography (CT) artifactreduction, the method comprising: receiving, by an estimated groundtruth apparatus, an initial CT image, the initial CT image comprising amajor artifact; generating, by an artifact reduction circuitry, anintermediate CT image based, at least in part, on the initial CT image,the intermediate CT image comprising a reduced artifact; and generating,by the estimated ground truth apparatus, an estimated ground truth imagebased, at least in part, on the intermediate CT image, the estimatedground truth image to be used for training an artificial neural network,adding, by a feature addition circuitry, a respective feature to each ofa number, N, copies of the estimated ground truth image to create thenumber, N, initial training images; generating, by a CT simulationcircuitry, a plurality of simulated training CT images based, at leastin part, on at least some of the N initial training images, each of atleast some of the plurality of simulated training CT images containingat least one respective simulated artifact; and training, by aconvolutional neural network (CNN) training circuitry, a CNN based, atleast in part, on the simulated training CT images and based, at leastin part, on the initial training images.
 2. The method of claim 1,further comprising at least one of filtering and/or denoising, by afiltering/denoising circuitry, the intermediate CT image, the filteredand/or denoised intermediate CT image corresponding to the estimatedground truth image.
 3. The method of claim 2, wherein thefiltering/denoising comprises one or more of low-pass filtering,segmentation and/or regional averaging of the intermediate CT image togenerate the estimated ground truth image.
 4. The method of claim 1,wherein the artifact reduction circuitry is metal artifact reductioncircuitry and the generating the intermediate CT image comprisesperforming metal artifact reduction.
 5. The method of claim 1, whereinthe initial CT image is an actual CT image of an imaging object thatcontains a metal implant.
 6. The method of claim 1, wherein the initialCT image is simulated based, at least in part, on a phantom image.
 7. Aconvolutional neural network (CNN) training system comprising: anestimated ground truth apparatus configured to receive an initial CTimage; an artifact reduction circuitry configured to generate anintermediate computed tomography (CT) image based, at least in part, onthe initial CT image, the initial CT image comprising a major artifact,the intermediate CT image comprising a reduced artifact; the estimatedground truth apparatus, further configured to generate an estimatedground truth image based, at least in part, on the intermediate CTimage, the estimated ground truth image for training an artificialneural network; a feature addition circuitry configured to add arespective feature to each of a number, N, copies of the estimatedground truth image to create the number, N, initial training images; aCT simulation circuitry configured to generate a plurality of simulatedtraining CT images based, at least in part, on at least some of the Ninitial training images, each of at least some of the plurality ofsimulated training CT images containing at least one respectivesimulated artifact; and a convolutional neural network (CNN) trainingcircuitry configured to train a CNN based, at least in part, on thesimulated training CT images and based, at least in part, on the initialtraining images.
 8. The system of claim 7, further comprising afiltering/denoising circuitry configured to at least one of filterand/or denoise the intermediate CT image, the filtered and/or denoisedintermediate CT image corresponding to the estimated ground truth image.9. The system of claim 8, wherein the filtering/denoising circuitry isconfigured to perform one or more of low-pass filtering, segmentationand/or regional averaging of the intermediate CT image to generate theestimated ground truth image.
 10. The system of claim 7, wherein theartifact reduction circuitry is metal artifact reduction circuitryconfigured to perform metal artifact reduction.
 11. The system of claim7, wherein the initial CT image is an actual CT image of an imagingobject that contains a metal implant.
 12. The system of claim 7, whereinthe initial CT image is simulated based, at least in part, on a phantomimage.
 13. The system of claim 7, wherein the feature addition circuitryis configured to extract a number, M, training patches from each of atleast some of the N initial training images, the simulated training CTimages generated based, at least in part, on at least some of the Mtraining patches.